REGULATORY FOCUS AND DECISION - MAKING 1 Chronic Motivational State Interacts with Task Reward Structure in Dynamic Decision - Making

Research distinguishes between a habitual, model-free system motivated toward immediately rewarding actions, and a goal-directed, model-based system motivated toward actions that improve future state. We examined the balance of processing in these two systems during state-based decision-making. We tested a regulatory fit hypothesis (Maddox & Markman, 2010) that predicts that global trait motivation affects the balance of habitualvs. goal-directed processing but only through its interaction with the task framing as gain-maximization or lossminimization. We found support for the hypothesis that a match between an individual’s chronic motivational state and the task framing enhances goal-directed processing, and thus state-based decision-making. Specifically, chronic promotion-focused individuals under gain-maximization and chronic prevention-focused individuals under loss-minimization both showed enhanced state-based decision-making. Computational modeling indicates that individuals in a match between global chronic motivational state and local task reward structure engaged more goaldirected processing, whereas those in a mismatch engaged more habitual processing. Keywords: motivation, decision-making, regulatory fit, regulatory focus, reward Correspondence concerning this article should be addressed to Jessica Cooper, Department of Psychology, The University of Texas, 108 E. Dean Keeton Stop A8000, Austin, TX 78712. Email: [email protected] Phone: (512) 232-2883 Fax: (512) 471-6175 REGULATORY FOCUS AND DECISION-MAKING 2 Chronic Motivational State Interacts with Task Reward Structure in Dynamic Decision-Making 1. Introduction Motivation is a key feature of decision-making that is often studied in terms of approaching positive states and avoiding negative states (e.g. Atkinson, 1964, Bandura, 1986; Roseman, Spindel, & Jose, 1990). We, along with others (e.g., Braver et al., 2014; Maddox & Markman, 2010), argue that the most common definition of motivation as a simple increase in effortful cognitive processing (i.e., trying harder) is outdated, and that a deeper understanding of the complex motivation-cognition interface is crucial to theorizing about motivation as well as cognition (see Braver et al., 2014, for a review). Under more recent views, motivation is thought to operate at multiple levels, and the effects of motivation on behavior derive from the interactions between these levels. The interactive nature of motivation on behavior is captured by the notion of “regulatory fit” (Higgins, 2000; Maddox and Markman, 2010), which is achieved when the individual’s global motivational state (chronic or situational) aligns with the local motivational task framing. Importantly, approach or avoidance motivation at one level can have vastly different effects on behavior depending upon the valence of motivation at another level. To date, little work has explored these multi-level motivational effects on the balance of cognitive processing. This is the focus of the present report. Regulatory fit effects have been shown in a variety of domains including judgments of morality (Camacho, Higgins, & Luger, 2003), communication effectiveness (Aaker & Lee, 2001; Cesario, Grant, & Higgins, 2004), and generation of anagram solutions (Shah et al., 1998). Unfortunately, no strong mechanistic explanations for these regulatory fit effects have been offered, mainly because these tasks are ones for which no unique optimal strategy can be REGULATORY FOCUS AND DECISION-MAKING 3 defined. This shortcoming has been addressed by examining tasks for which the optimal strategy is uniquely identifiable, and importantly is mediated by a specific cognitive process. This work tests the hypothesis that a “fit” between the global and local motivational state enhances effortful cognitive processing at the expense of automatic habitual processing (See Maddox & Markman, 2010, for a review). Critically, whether this enhanced effortful processing leads to better performance depends upon whether optimal task performance is mediated by effortful processing. Thus, this work argues that the motivation-cognition relationship involves a threeway interaction between the global motivational state, the local motivational state, and the cognitive processing system that optimally mediates task performance. When the task is one for which optimal performance requires effortful processing, a regulatory fit is advantageous. However, when the task is one for which optimal performance requires automatic habitual processing, such as implicit category learning (Grimm, Markman, Maddox & Baldwin, 2008), a regulatory mismatch is advantageous. Tests of this three-way interaction find support in studies that examine category learning (e.g. Grimm et al., 2008; Maddox, Baldwin & Markman, 2006) and decision-making (Worthy, Maddox & Markman, 2007; Otto, Markman, Gureckis & Love, 2010). Regulatory fit effects in decision-making have shown that decision-makers in a regulatory fit more often choose to systematically explore their environment, while those in a regulatory mismatch more often exploit the highest-valued option (Otto et al., 2010; Worthy et al., 2007). Worthy et al. (2007) and Otto et al. (2010), like much work on regulatory fit, focused on the effect of situational (or induced) regulatory focus. Situational or experimentally-induced motivational focus, obtained by making individuals temporarily experience either a subjective history of promotion success or prevention success (Higgins et al., 2001), is extremely helpful in REGULATORY FOCUS AND DECISION-MAKING 4 providing methods for boosting overall task performance, but does little to identify performance advantages related to stable traits of the individual. Chronic promotion and prevention focus is measured using the Regulatory Focus Questionnaire (RFQ; Higgins et al., 2001), which provides scores for two motivational modes that affect the sensitivity of the motivational system: a promotion focus, in which one becomes more sensitive to potential gains, or a prevention focus, in which one becomes more sensitive to potential losses (Higgins, 1997). Here, we examine whether these chronic (trait-driven) dispositional tendencies lead people to engage in qualitatively different decision-making strategies depending on task framing, whether the goal of the task is to maximize gains or minimize losses. Simply altering the way that decision-making tasks are framed is predicted to drastically alter the types of decision-making strategies that people implement, and could provide a simple mechanism for optimizing performance in decision-making tasks by catering the framing to an individual’s chronic motivational mode. 1.1. The Current Study In the current work, we test the hypothesis that chronic (trait-driven) motivational modes interact with reward structure to affect decision-making performance and cognitive processing by evaluating decision-making performance and utilizing computational modeling to quantify the degree to which participants use goal-directed versus habitual strategies. Habitual and goaldirected systems are often referred to as model-free and model-based in the neuroscience literature (e.g. Daw, Niv & Dayan, 2005; Doya, Samejima, Katagiri & Kawato, 2002; Gläscher, Daw, Dayan & O’Doherty, 2010). The model-free, or habitual, system is motivated toward actions that lead directly to reward while the model-based, or goal-directed, system is more computationally demanding and requires consideration of how actions can affect both immediate and future outcomes. The distinction that use of the goal-directed system is more REGULATORY FOCUS AND DECISION-MAKING 5 computationally demanding has been reflected in several recent studies that have found a relationship between goal-directed strategies and working memory processes (e.g. Gershman, Markman & Otto, 2014; Otto, Gershman, Markman & Daw, 2013). The decision-making task that we utilize has been widely used to evaluate state-based decision-making (e.g. Cooper, Worthy, Gorlick & Maddox, 2013; Gureckis & Love, 2009a; Worthy, Cooper, Byrne, Gorlick & Maddox, 2014). In this task participants repeatedly choose between two rewarding options and gain information about the reward environment by making decisions and receiving rewards. One option always provides a larger immediate reward but causes the rewards available on future trials to decrease. The other option provides a lower immediate reward on each trial but causes the available future rewards to increase by improving one’s future state. Thus, options that favor the goal-directed system are directly pitted against those that favor the habitual, reward-based system. Importantly, this task is amenable to computational modeling that can be used to quantify strategy engagement, allowing us to directly link the balance of processing with differences in performance (Worthy et al., 2014). We fit participants’ data with a series of computational models, including a model that includes a free parameter that quantifies the weight placed on the output from goal-directed or habitual processing systems. We predict that a regulatory fit, known to shift the cognitive balance toward goal-directed processing (e.g. Markman et al. 2007; Worthy et al. 2007) and to aid complex problem-solving (e.g. Grimm et al., 2008), facilitates the use of goal-directed processes. Specifically, we predict that individuals in a match between chronic regulatory-focus and reward structure will be more motivated toward improvements in state than those in a mismatch. We expect that a regulatory REGULATORY FOCUS AND DECISION-MAKING 6 match will result in differences in performance and in computational model parameter estimates reflecting the increased use of more computationally-demanding, goal-directed strategies. We test the prediction that chronic regulatory focus interacts with the task reward structure in two versions of the decision-making task. In one version participants are asked to minimize losses, while in the other they are asked to maximize gains. Importantly, the underlying reward structure, optimal strategy, and interaction between the reward options is identical between these conditions—the only difference is that all of the reward values in the loss-minimization condition are shifted by a constant so that they are negative and provide losses on each trial. We predict that in the gain-maximization condition, individuals who are more promotion-focused will experience a regulatory match and will perform better than those who are prevention-focused. Additionally, in the loss-minimization condition we predict that prevention-focused individuals will experience a regulatory match and will outperform promotion-focused individuals. 2. Methods 2.1. Participants Participants were 80 young adults (Mage = 18.5, SD = .64) recruited from the University of Texas, with 20 participants in each group. Each participant was compensated $10 per hour for his or her participation. Informed consent was obtained from all participants, and the experiment was approved for ethics procedures using human participants at the University of Texas. 2.2. Materials and Procedure Students completed the Regulatory Focus Questionnaire (RFQ; Higgins et al., 2001) as part of the introductory to psychology participant screening. Prevention and promotion focus scores were calculated for each individual according to scoring instructions. A relative REGULATORY FOCUS AND DECISION-MAKING 7 promotion-focus score was calculated for each participant by subtracting his or her preventionfocus score from his or her promotion-focus score. Individuals with a positive relative promotion-focus score were classified as promotion focused, while individuals with a prevention-focus score higher than their promotion-focus score (negative relative promotionfocus score) were classified as prevention focused. Promotion focused and prevention focused individuals were invited to our lab to complete the study. Upon arriving, participants were screened again with the RFQ to ensure that their scores were consistent. Participants who had equal promotion and prevention focus scores were not included in our study, as they did not fit into either group. Participants were not informed of their promotion or prevention focus status. Demographic information for each group can be observed in Table 1. Participants completed the Mars Farming task in one of two conditions: gainmaximization or loss-minimization. The task was completed on PC computers using Psychtoolbox 2.54 for MATLAB (Brainard, 1997; Pelli, 1997). Each task consisted of 250 trials in which participants would choose between two reward options. Table 1: Demographic information Gain-maximization Loss-minimization Promotion Prevention Promotion Prevention Age 18.67 (.68) 18.82 (.60) 18.38 (.87) 18.47 (.52) Gender 10 F / 10 M 14 F / 6 M 12 F / 8 M 14 F / 6 M RFQ Difference 5.45 (2.74) -3.80 (2.07) 5.10 (3.43) -4.50 (1.64) Note: Standard deviations in parenthesis. RFQ difference score is promotion focus score minus prevention focus score; positive values indicate higher promotion focus, and negative indicate higher prevention focus. 2.2.1. Gain-maximization In the gain-maximization condition, participants were asked to select one of two systems for extracting oxygen on each trial. Following each choice, a small tank filled with the oxygen REGULATORY FOCUS AND DECISION-MAKING 8 extracted on that trial. This was then transferred to a larger tank. A line on the larger tank corresponded to the amount of oxygen needed to sustain life on Mars, and participants were told to collect at least that much oxygen over the course of the experiment. A sample screen shot from the gain-maximization condition is displayed in Fig. 1A (left panel). Fig. 2A (top panel) shows the reward structure for the gain-maximization task. Participants were not told about the reward contingencies of the two options and had to learn about them through trial and error. The reward associated with selecting each option is dependent on the number of times that one of the options, referred to as the increasing option, has been selected over the previous 10 trials (the state). For example, if the increasing option had been selected on four of the previous 10 trials, then 35 units of oxygen would be extracted with the increasing option system, whereas 75 units of oxygen would be extracted with the decreasing option system. Choosing the increasing option causes the gain for both options to increase on future trials whereas choosing the other option, referred to as the decreasing option, causes the gain for both options to decrease on future trials. However, the decreasing option always gives a higher gain on any given trial, as suggested by the example above. Thus, on every trial the decreasing option will yield a higher gain, but the more that the increasing option is selected the larger the gain obtained for both options will be on future trials. If participants select the increasing option on each trial, they would eventually reach the highest state (10), whereas selecting the decreasing option on each trial would eventually lead to the lowest state (0). Selecting the increasing option involves a cost in the immediate gain but a delayed benefit in overall gain, whereas selecting the decreasing option involves a benefit in the immediate gain but a delayed cost in overall gain. The goal line was drawn on the oxygen REGULATORY FOCUS AND DECISION-MAKING 9 collection tank (Figure 1A). The positioning of this line corresponded to performance that would be attained by selecting the optimal choice (increasing option) on approximately 80% of trials A) B) Figures 1A and 1B: Screen shots from the experiment. Figure 1A (left) shows a screen shot from the gain-maximization condition. Figure 1B (right) shows a screen shot from the lossminimization condition. A) 1 With full knowledge of task length an ideal actor would select the increasing option until the last 8-9 trials, then switch to the decreasing option. Participants were not provided information regarding trial number, and our analysis will treat the increasing option as the optimal choice on all trials. REGULATORY FOCUS AND DECISION-MAKING 10